Approach to machine learning for extraction of real-world data variables from electronic health records.

Background: As artificial intelligence (AI) continues to advance with breakthroughs in natural language processing (NLP) and machine learning (ML), such as the development of models like OpenAI's ChatGPT, new opportunities are emerging for efficient curation of electronic health records (EHR) into real-world data (RWD) for evidence generation in oncology. Our objective is to describe the research and development of industry methods to promote transparency and explainability. Methods: We applied NLP with ML techniques to train, validate, and test the extraction of information from unstructured documents (e.g., clinician notes, radiology reports, lab reports, etc.) to output a set of structured variables required for RWD analysis. This research used a nationwide electronic health record (EHR)-derived database. Models were selected based on performance. Variables curated with an approach using ML extraction are those where the value is determined solely based on an ML model (i.e. not confirmed by abstraction), which identifies key information from visit notes and documents. These models do not predict future events or infer missing information. Results: We developed an approach using NLP and ML for extraction of clinically meaningful information from unstructured EHR documents and found high performance of output variables compared with variables curated by manually abstracted data. These extraction methods resulted in research-ready variables including initial cancer diagnosis with date, advanced/metastatic diagnosis with date, disease stage, histology, smoking status, surgery status with date, biomarker test results with dates, and oral treatments with dates. Conclusion: NLP and ML enable the extraction of retrospective clinical data in EHR with speed and scalability to help researchers learn from the experience of every person with cancer.

Financial barriers and opportunities for conservation adoption on U.S. rangelands: A region-wide, ranch-level economic assessment of NRCS-sponsored Greater Sage-grouse habitat conservation programs.

Sagebrush ecosystems of the western U.S. support ranching livelihoods and imperiled populations of the Greater Sage-grouse (Centrocercus urophasianus). Incentive-based conservation such as cost-sharing is the primary tool used by the federal government to support conservation practices on rangelands in the U.S. Financial support for adopting specific prescribed grazing practices on private land has been supported through the U.S. Department of Agriculture Natural Resources Conservation Service (NRCS)-led Sage-Grouse Initiative (SGI), initiated in 2010 as an unparalleled private and public effort to conserve Greater Sage-grouse habitat. The purpose of this research was to provide an economic assessment of the impact of this conservation program on participating ranches. Representative ranch enterprise budgets and ranch economic models were created for this analysis for eleven NRCS Major Land Resource Areas where critical sage-grouse habitat exist, including parts of Idaho, Montana, Nevada, Oregon, Washington, and Wyoming. Results of the economic assessment showed that SGI/NRCS financial support alleviated the financial impact of conservation practice adoption, but negative financial impacts were estimated in some locations and more frequently for smaller ranches. Larger ranches were found to do better under these programs on average. Results demonstrate the important role of research and government financial support in removing financial barriers to conservation adoption on rangelands.

Expected Net Benefit of Vaccinating Rangeland Sheep against Bluetongue Virus Using a Modified-Live versus Killed Virus Vaccine.

Recurring outbreaks of bluetongue virus in domestic sheep of the US Intermountain West have prompted questions about the economic benefits and costs of vaccinating individual flocks against bluetongue (BT) disease. We estimate the cost of a BT outbreak on a representative rangeland sheep operation in the Big Horn Basin of the state of Wyoming using enterprise budgets and stochastic simulation. The latter accounts for variability in disease severity and lamb price, as well as uncertainty about when an outbreak will occur. We then estimate the cost of purchasing and administering a BT vaccine. Finally, we calculate expected annual net benefit of vaccinating under various outbreak intervals. Expected annual net benefit is calculated for both a killed virus (KV) vaccine and modified-live virus vaccine, using an observed price of $0.32 per dose for modified-live and an estimated price of $1.20 per dose for KV. The modified-live vaccine's expected annual net benefit has a 100% chance of being positive for an outbreak interval of 5, 10, or 20 years, and a 77% chance of being positive for a 50-year interval. The KV vaccine's expected annual net benefit has a 97% chance of being positive for a 5-year outbreak interval, and a 42% chance of being positive for a 10-year interval. A KV vaccine is, therefore, unlikely to be economically attractive to producers in areas exposed less frequently to BT disease. A modified-live vaccine, however, requires rigorous authorization before legal use can occur in Wyoming. To date, no company has requested to manufacture a modified-live vaccine for commercial use in Wyoming. The KV vaccine poses less risk to sheep reproduction and less risk of unintentional spread, both of which facilitate approval for commercial production. Yet, our results show an economically consequential tradeoff between a KV vaccine's relative safety and higher cost. Unless the purchase price is reduced below our assumed $1.20 per dose, producer adoption of a KV vaccine for BT is likely to be low in the study area. This tradeoff between cost and safety should be considered when policymakers regulate commercial use of the two vaccine types.

Cattle producers' economic incentives for preventing bovine brucellosis under uncertainty.

Cattle in the Greater Yellowstone Ecosystem occasionally contract bovine brucellosis from free-ranging elk and bison. Cattle producers use a variety of brucellosis prevention activities to reduce their herds' risk of contracting brucellosis, such as: (1) having state agency personnel haze elk off private land, (2) fencing haystacks, (3) administering adult booster vaccination, (4) spaying heifers, (5) altering the winter-feeding schedule of cattle, (6) hiring riders to prevent cattle-elk commingling, and (7) delaying grazing on high-risk allotments. Their brucellosis prevention decisions are complicated, however, by several sources of uncertainty, including the following: a cattle herd's baseline risk of contracting brucellosis, the inherent randomness of brucellosis outbreaks, the cost of implementing prevention activities, and the activities' effectiveness. This study eliminates one source of uncertainty by estimating the cost of implementing brucellosis prevention activities on a representative cow/calf-long yearling operation in the southern GYE. It then reports the minimum level of effectiveness each prevention activity must achieve to justify investment by a risk-neutral producer. Individual producers face different levels of baseline risk, however, and the US government's brucellosis-response policy is constantly evolving. We therefore estimate breakeven levels of effectiveness for a range of baseline risks and government policies. Producers, animal health experts, and policymakers can use this study's results to determine which brucellosis prevention activities are unlikely to generate sufficient expected benefits to cover their cost of implementation. Results also demonstrate the influence of government policy on producers' incentives to prevent brucellosis. Policies that increase the magnitude of economic loss a producer incurs when their herd contracts brucellosis subsequently decrease prevention activities' breakeven levels of effectiveness, and increase producers' incentives to implement those activities. Producers' incentives to implement prevention activities also increase as activities' costs decrease. Policymakers can easily adapt the results of this analysis to help target cost-share agreements to producers and prevention activities most likely to generate positive expected net benefits. Epidemiologists can also use our results to help prioritize future research on the technical effectiveness of various brucellosis prevention activities.